Skip to content

Flattening CLUBB's grid: allocations by any name, bounds by component, generics by arity - #27

Open
chenyueqi wants to merge 78 commits into
tier0-gapsfrom
tier1-grid
Open

Flattening CLUBB's grid: allocations by any name, bounds by component, generics by arity#27
chenyueqi wants to merge 78 commits into
tier0-gapsfrom
tier1-grid

Conversation

@chenyueqi

@chenyueqi chenyueqi commented Sep 4, 2026

Copy link
Copy Markdown
Member

Stacked on #25. What the flat plan, the gates and the emitter needed to take type(grid) :: gr and its neighbours through, found on the CLUBB extension's tier-1 units.

Flattening

  • Allocations: allocate( gr%zm(ngrdcol,gr%nzm), gr%zt(...), … ) — one statement, many components, the object named by the setup routine's dummy rather than this.
  • Bounds: an axis sized by a component of the same or another object (gr%nzm) is spelled by that component's flat name and the component is carried as an input; a dummy declared thvm(ngrdcol, gr%nzt) the same; an allocation sized by the allocating routine's dummy (1:nz) by the planned subprogram's dummy of that name; a bound from one is its upper bound alone.
  • ngrdcol is declared once when the subprogram already takes it.
  • Generics: a call through a generic is followed into the specifics its arity fits; the recursion guard is a stack, not a memo.
  • Private module state (error_code % clubb_debug_level) is left to the module, under left_to_module.
  • An intent(out) object's written components are read off the object the translation returned.

Read/write scope: a sibling's generics are procedures to it; a sibling's buffer-OUT actuals are read; a stub under control flow (if (stats%l_sample) then / call stats_update / end if) is a stub block.

Two silent translation defects, both fixed with executed tests

  • A DO loop's index read after the loop: Fortran leaves it one step past the end on completion, for leaves the last value. lscale_width_vert_avg integrated one level short. A loop whose index a later statement reads before anything redefines it gets a for … else with the completion value; an EXIT (break) skips it, as Fortran keeps the exit value.
  • ep = Rd / Rv over two real parameters was folded with integer division (the old "no real literal means integer arithmetic" rule) and came out zero; every unit reading ep1/ep2 computed NaN. The resolver records each constant's declared base type and the fold types names by it.

Bit parity: SUM accumulates in element order (_f_vsum beside _f_vdot); np.sum's pairwise rounding drifted 12 ULP in vertical_integral.

Gate: a subprogram the operator declared ungated is not sampled; a translated bare error stop (SystemExit) is a failed comparison, not the end of the run.

Tier-2 replay (recorded runs through dump-replay), found on CLUBB's advance_* solvers replayed on bomex recordings:

  • The dump parser took only numbers: a logical T/F header (l_implemented) was no value and a zero-extent component (scalar tracers under sclr_dim = 0) vanished with its empty body. Both are values.
  • The recorder wrote what the objects received; the plain OUT/INOUT dummies (wp2, wp3, mono_flux_limiter's low_lev_effect) are outputs too.
  • Two specifics of one arity (tridiag_solve with its optional rcond and the multiple-rhs one without) were picked by count; the ranks of the bare-name actuals decide.
  • A component name on an OUT actual (pdf_params%chi_1) was counted a read of a variable of that name — the documented disagreement inherited from the pipeline, resolved the tidier way now that a translation showed it (test flipped, deliberately).
  • A call to a procedure of a stubbed module that no statement stub answers (lapack_band_solvex) deferred its whole block, if ( method == lapack ) included, so the candidate raised on the LU path the run takes. It raises on its own line.
  • max(2, edsclr_dim) sizing a local is a bound Python can spell; a comma in a bound is legal inside max/min alone.
  • An optional OUT handed on (rcond = rcond) carries the caller's presence (want_rcond=want_rcond, not True) — xm_wpxp_solve took the LAPACK diagnostic path on every call — and the read/write gate counts that hand-on as a read of the optional on both sides.
  • A module-state object handed whole to a companion function (sponge_layer_damping's profile into sponge_damp_xm) is followed under the callee's dummy, so the plan carries the components the callee reads; a derived-type state variable is no longer listed as "left to the module".
  • A local parameter initialized by a scalar times an array constructor is refused at emission #26: 100._core_rknd * (/ ... /) as a local parameter — each constructor rendered where it stands, the rest through the token pass, elementwise (saturation; pdf_closure's hybrid-PDF path).
  • A subroutine with no OUT argument returns zero values, not one (finalize_tau_sponge_damp_api); a declared-ungated subprogram is not compared on a recording either, and the reason is reported.

Merged with main (2026-09-04, after PRs #29 and #30 landed there): main's comparison loop is kept as is — a translated ERROR STOP or an IndexError is a draw redrawn — with this branch's recorded-sample path, zero-value return and declared-ungated skip beside it; constant folding carries main's parameter aliases together with this branch's intrinsic calls; the appended tests of both sides are kept. Merged bottom-up through #22 and #25 so each PR stays mergeable.

Held to (self-check 2026-09-04 before tier 3): full suite and mypy green; conformance/ --plugin-set recast-clubb green; the corpus re-run on this branch regresses no unit (numfor's genrand 7→3 disagreeing blocks, slsqp 12→8, csplines/fitpack/slsqp_core as before); tools/emit_diff.py against the translator goes from 34 differing subprograms at this branch's base to 48, every added row an intended class — _f_vsum for SUM, want_x=want_x where an optional OUT is handed on (wvsat's qsat*: the translator's True is the defect fixed here), declared-initializer locals, and define_swept_areas where the translator drops the reshape around a constructor and this branch keeps it (#26). CLUBB tier 0 stays 12/14; tier 1 is 7/11 bit-exact with gr flattened (advance_helper_module: 25 subprograms, 7,770 points); tier 2 (summaries/tier2.json in clubb-jax): 10 of 12 units bit-exact on 20–40 recorded calls each from bomex and cgils_s6 under the hybrid PDF — advance_wp2_wp3, advance_xm_wpxp, advance_xp2_xpyp, advance_xp3, advance_windm_edsclrm, clip_explicit, mono_flux_limiter, mixing_length (incl. diagnose_lscale_from_tau), pdf_closure, new_hybrid_pdf_main — plus the sampled plain routines of clip_explicit, mixing_length and pdf_closure. sponge_layer_damping compares nothing (initializer's below-layer levels are undefined on both sides, declared; finalizer has no output; the sponge_damp_* functions take gr and have no flat plan); new_pdf_main has no recording (the new PDF needs LES input fields).

Tier 3 (2026-09-04): advance_clubb_core — the whole CLUBB time step, 135 dummies flattened to 476 arguments over 7 derived dummies and the 14 sponge state objects — is bit-exact on 40 recorded steps (20 bomex, 20 cgils_s6 under the hybrid PDF): 453,700 compared values, no rewrites to notarize. Two more fixes got it there: a probe spans blank and comment lines inside a continued call (cpp leaves them where an #ifdef stood), and a generic function reference is ranked by the specific it dispatches to (a nested sat_mixrat_liq_api( ..., thlm2T_in_K_api( ... ) ) deferred a block, and a deferred block took the subprogram out of the gate).

🤖 Generated with Claude Code

https://claude.ai/code/session_01Cne4hrGgYqhTMzVgzJtXYd

chenyueqi and others added 21 commits September 3, 2026 20:22
CLUBB's grid: setup_grid allocates every component in one statement,
allocate( gr%zm(ngrdcol,gr%nzm), gr%zt(ngrdcol,gr%nzt), ... ), naming the
object by its dummy rather than this, sizing an axis by another component
of the same object and another by the module's private parameters. The
allocation reader takes any object name and every component of a
statement; a bound spelling obj%comp is bound to that component's flat
name, and the component is carried as an input whether or not the body
reads it; a bound from one is spelled by its upper bound alone; and the
driver's extent is declared once when the subprogram already takes it as a
dummy, as CLUBB's do with ngrdcol.

The read/write scope also learns a sibling's generics
(companion_externals): a call spells zt2zm_api, and a name the scope did
not know as a procedure it counted as a read of data.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cne4hrGgYqhTMzVgzJtXYd
…ad; a stub under control flow is a stub block

calc_pressure declares thvm(ngrdcol, gr%nzt): a dummy sized by a component
of the object it takes. The flat signature spells that component's flat
name, an argument of the adapter, and the plan carries the component as an
input whether or not the body reads it -- also when another object's
allocation names it (sponge_layer_damping's profile, sized by gr%nzm).
Every dummy's type is read before the objects are walked so the reference
resolves whichever order they come in.

Skx_func(..., Skw_zm) into a sibling: the caller's storage is passed in and
returned, so the caller reads the actual as well as writing it. The
companion table now says which positions are buffers, and the read/write
scope reads them, the rule it already applied to its own subprograms.

if ( stats%l_sample ) then / call stats_update(...) / end if is a
framework stub under a condition. The condition is read on both sides and
the only disagreement is the stub's actuals; the block is waived like a
bare stub, and the verdict names the calls.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cne4hrGgYqhTMzVgzJtXYd
…ate state left to its module, ungated means not sampled

An object passed to a sibling through a generic (zt2zm_api over
grid_class's specifics) was not followed, so the adapter built a record
missing the components the specific reads. A generic is followed into
every specific; the union is what the adapter carries.

An allocation sized by the allocating routine's dummy
(coef_wp4_implicit(1:ngrdcol,1:nz)) is spelled by the planned
subprogram's dummy of the same name, an argument of the adapter.

A module variable the module keeps private (error_code's
clubb_debug_level) reaches no use statement, so no adapter sets it; both
sides run with the module's default and the plan records it under
left_to_module instead of failing.

A subprogram the operator declared ungated is not sampled by the
bit-exact gate either: the declaration says the reference cannot be held
on generated inputs, and rcm_sat_adj proves it by error-stopping the run.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cne4hrGgYqhTMzVgzJtXYd
…rsion guard is a stack

Following every specific with the caller's positional actuals mapped the
object onto whichever dummy sat at that position -- zm_min for the
one-level specific -- and the specific the call actually reaches was then
skipped, because a nested follow had marked it visited first. The
specifics whose argument count fits the call are followed (the exact
count when any matches), and a key is released when its follow returns:
the set guards against cycles, it is not a memo.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cne4hrGgYqhTMzVgzJtXYd
…ct is read off what the translation returned; a translated error stop is a failed comparison

CLUBB's constants_clubb declares ep = Rd / Rv over two real parameters.
The fold's rule -- an initializer with no real literal is integer
arithmetic throughout -- made it zero, and every unit reading ep1 or ep2
computed NaN. The resolver now records each constant's declared base
type, the fold types a name by it, and both renderers carry the
environment along in dependency order; nrk = runge_kutta_type / 10 stays 4.

The Python flat adapter read a written component off the record it built
from the inputs even when the dummy was intent(out) and the translation
returned a new object (sponge_layer_damping's profile): it now rebinds to
the returned object first.

A translated bare error stop raises SystemExit, which the bit-exact gate
let through and which ended the run with no report; it is now a
comparison that could not be made, named as such.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cne4hrGgYqhTMzVgzJtXYd
CLUBB's lscale_width_vert_avg searches with do k_avg_upper = k, ...; if
(...) exit; end do and integrates up to k_avg_upper. On completion Fortran
leaves the index one step past the end, m1 + n * m3; Python's for leaves
the last value, and the integral ran one level short -- a NaN on one side
in wp23_term_splat_lhs. For a loop whose index some later statement of the
subprogram names, the emitter adds a for-else that sets the completion
value; an EXIT is a break and skips it, as Fortran keeps the exit value.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cne4hrGgYqhTMzVgzJtXYd
gfortran's inlined SUM is a loop in element order; np.sum pairs its terms
and rounds differently -- CLUBB's vertical_integral drifted 12 ULP on 3
of 7770 points. _f_vsum, beside _f_vdot, walks the array in Fortran order
or along the named axis, and the intrinsic table points SUM at it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cne4hrGgYqhTMzVgzJtXYd
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cne4hrGgYqhTMzVgzJtXYd
…ned; the corpus re-baselined

An else on every loop whose variable any later statement names put one on
most loops of a module -- i and k are reused by the next loop -- and moved
the translator differential from 33 to 130 differing subprograms. A later
DO over the same variable or an assignment to it redefines it first and
closes the question; a later loop's bounds may still read it and count.
The differential is at 45, every one of the twelve new differences an
intended class: an array-valued .or. spelled |, SUM through _f_vsum, the
completion value on a search loop.

The corpus, re-run on this branch: no unit regresses; csplines, fitpack and
slsqp_core have fewer read/write disagreements; numfor's sorting reaches
the bit-exact gate for the first time (its specifics are private behind a
public generic) and exposes a bare use-imported constant in a local
parameter, issue #28.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cne4hrGgYqhTMzVgzJtXYd
…s the run owns

A character component (err_info%err_header) has no flat spelling and the
physics reads it only to print: left at the object's default, under
left_to_module. A derived-type module variable declared in a list over
several lines (the four sponge settings) is found on the module record
when the one-line pattern misses it. An allocation sized by the allocating
routine's dummy when the planned subprogram has none of that name
(coef_wp4_implicit(1:ngrdcol,1:nz) under advance_wp2_wp3, which takes
nzm and nzt) becomes an integer argument of the adapter, extent_args,
which the recorder writes from size() and the sampled gate sizes like any
scalar an array's dims name. Every advance_* plan of CLUBB is usable on
this.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cne4hrGgYqhTMzVgzJtXYd
…es it

CLUBB passes ngrdcol as a dummy of every subroutine the recorder probes;
a local of the same name was a second declaration, and gfortran said so
forty times.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cne4hrGgYqhTMzVgzJtXYd
CLUBB continues its calls as a, b, & ! In on every line: the comment after
the ampersand left a blank the joiner did not strip, and the probe carried
& into its argument list.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cne4hrGgYqhTMzVgzJtXYd
CLUBB allocates its scalar-tracer coefficients only when sclr_dim > 0;
under bomex it is 0, and reshape of an unallocated component faulted the
probed run after three dumps. An allocatable or pointer component is
recorded inside allocated()/associated(), as a zero-extent record
otherwise, and the extent argument taken from its size() is zero when it
was never allocated.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cne4hrGgYqhTMzVgzJtXYd
…eads and writes nothing

The read/write scope marked whatever sat at a union of positions as
written when a call reached a sibling's generic whose specifics differ in
arity: CLUBB's tridiag_solve and zm2zt_api marked nzt, l_implemented and
gr. The companion table now carries each specific with its argument
count, and the scope picks the one the actuals fit.

A name use-imported from a stubbed module is a call the translation
answers with a stub -- pass, for stats_update -- so the source side no
longer counts its actuals: stats_tmp and the budget names were reads only
that side saw.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cne4hrGgYqhTMzVgzJtXYd
The first cut put every name imported from a stubbed module into the
externals table, and a bare name in that table is not a read on the source
side: every constant of constants_clubb -- a stub and a table of constants
-- became a read only the target saw. The frontend now asks the stub
module's own record which imports are procedures (an absent stub is taken
at its import list), and hands the list to the translation, whose protocol
lists them as procedures so a call to a stand-in function is a call and not
a read of its name.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cne4hrGgYqhTMzVgzJtXYd
…left off

The companion table said which positions a call writes and left the
reading to 'everything else', so an INOUT actual -- lhs, rhs, err_info into
CLUBB's band_solve -- was written and not read on the source side. Each
entry now says what the caller reads: IN, INOUT, UNKNOWN and a buffer OUT.
And a specific of a generic fits a call whose actuals fall between its
required and its total count, since an optional dummy may be left off.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cne4hrGgYqhTMzVgzJtXYd
call band_solve( ..., solut, rcond = rcond ) (CLUBB): bound by position the
keyword fell on the dummy before it and was counted a read; the companion
table now carries the dummies' names and the scope binds a keyword actual
by name, as it does for a callee of the module itself.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cne4hrGgYqhTMzVgzJtXYd
…ank-picked specifics, stubbed calls raise in place

Found by replaying CLUBB's advance_* solvers on bomex recordings:

* dump-replay's parser took only numbers: a logical ``T``/``F`` header
  (l_implemented) was no value, and a zero-extent component (scalar
  tracers under sclr_dim = 0) vanished with its empty body. Both are
  values now.
* The recorder wrote what the objects received; the plain OUT/INOUT
  dummies (wp2, wp3, mono_flux_limiter's low_lev_effect) are outputs too.
* Two specifics of one arity (tridiag_solve with its optional rcond and
  the multiple-rhs one without) were picked by count; the ranks of the
  bare-name actuals decide, so an IN logical is no longer scored written.
* A component name on an OUT actual (``pdf_params%chi_1``) was counted a
  read of a variable of that name; the tidier answer the pipeline's test
  kept as a documented disagreement is now the answer on both sides.
* A call to a procedure of a stubbed module that no statement stub
  answers (lapack_band_solvex on CLUBB's LAPACK path) deferred its whole
  block, condition and LU branch included, so the candidate raised on the
  path the run takes. It raises on its own line now.
* ``max(2, edsclr_dim)`` sizing a local is a bound Python can spell; a
  comma in a bound is legal inside max/min alone.
* The flat-plan module lookup tolerated a missing record (mypy).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cne4hrGgYqhTMzVgzJtXYd
``call inner( x, y, rc = rc )`` where ``rc`` is the caller's own optional
OUT was rendered ``want_rc=True``: present in the callee on every call.
CLUBB's xm_wpxp_solve hands ``rcond = rcond`` to band_solve, whose
``present(rcond)`` branch is the LAPACK diagnostic path -- taken, on the
replay, on every call the run made without rcond. The presence is the
caller's own sentinel for an optional OUT and ``is not None`` for an
optional IN.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cne4hrGgYqhTMzVgzJtXYd
The source side of the read/write gate scored call solve( ..., rc = rc )
as a write of the caller's own optional rc; the translation spells the
callee's present(rc) as the caller's want_rc sentinel, a read. Both
sides now count the read, for a sibling's procedure (companion externals
carry the optional OUT positions) and for one of this module.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cne4hrGgYqhTMzVgzJtXYd
…sion over a constructor is a value (#26)

* The flat plan followed only the caller's own dummies into callees. A
  module-state object passed whole to a derived-type dummy -- CLUBB's
  advance_xm_wpxp hands sponge_layer_damping's profile to sponge_damp_xm,
  which reads ``profile%tau_sponge_damp`` -- was left to the module, and
  the replay found it unallocated. It is followed under the callee's
  dummy now, so its components come back under the state's own name.
  A derived-type state variable is no longer listed as left to the
  module: it is the objects path's business.
* ``100._core_rknd * (/ ... /)`` as a local parameter (#26, saturation and
  pdf_closure's hybrid-PDF path): the token pass rendered a bare
  constructor and handed anything around one to the parser, whose
  literals were never hoisted. Each constructor is rendered where it
  stands and the rest goes through the token pass, elementwise.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cne4hrGgYqhTMzVgzJtXYd
chenyueqi and others added 8 commits September 4, 2026 13:09
CLUBB's finalize_tau_sponge_damp_api deallocates a component and returns;
its adapter returns None, which the gate counted as one value against
zero out-intent arguments.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cne4hrGgYqhTMzVgzJtXYd
The declaration carries the reason -- CLUBB's sponge initializer leaves
the levels below the layer undefined on both sides -- and the replay
reported it, then compared heap contents against np.empty anyway.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cne4hrGgYqhTMzVgzJtXYd
No unit regresses; numfor's genrand goes from 7 to 3 disagreeing blocks and
slsqp from 12 to 8 (the component-name and handed-on-optional rules).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cne4hrGgYqhTMzVgzJtXYd
The comparison loop takes main's: a translated ERROR STOP or an IndexError
on a draw is a draw the subprogram does not take, redrawn. This branch's
recorded-sample branch, zero-value return and declared-ungated skip sit
beside it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cne4hrGgYqhTMzVgzJtXYd
Against main's own baseline no unit regresses; slsqp goes from 15 to 7
disagreeing blocks of the same 106 checked.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cne4hrGgYqhTMzVgzJtXYd
cpp leaves blank lines where an #ifdef stood inside CLUBB's
advance_clubb_core call; the probe took one for the end of the statement.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cne4hrGgYqhTMzVgzJtXYd
CLUBB's advance_clubb_core passes thlm2T_in_K_api( ... ) -- a generic
over a scalar and a 2-D specific -- as an actual to sat_mixrat_liq_api.
Ranked scalar without looking, the outer generic matched no specific, its
block was deferred, and a deferred block takes the whole subprogram out of
the gate. A non-elemental function's rank is its declared result's.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cne4hrGgYqhTMzVgzJtXYd
…eftover CYCLE is delegated

CLUBB's interpolators cycle from inside an IF in a DO loop. The lowering
passed the continue through into a lax.cond branch -- a SyntaxError that
took the whole emitted module down. The loop body's continuation now moves
into the branches that do not cycle, before the fori_loop lowering; what
that cannot fold, and every break, delegates the subprogram to the host.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cne4hrGgYqhTMzVgzJtXYd
chenyueqi and others added 8 commits September 4, 2026 18:09
…read

advance_clubb_core's anchor unpacks what a solver hands back -- stats =
_out[0], pdf_params = _out[4] -- from the buffer the flat rewrite
elides at the call. The flat outputs bound to the actuals, so an object's
unpack is already true and goes; an array's is the actual it names.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cne4hrGgYqhTMzVgzJtXYd
The anchor unpacks _out[k] by position; an object's outputs are bound by
component and left their slot out, so advance_clubb_core's
err_info = _out[10] pointed past the list.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cne4hrGgYqhTMzVgzJtXYd
… its guard

advance_clubb_core calls numerical_check's parameterization check under
clubb_at_least_debug_level_api(2), false with statistics off; the check
takes character arguments and its port leaves it on the host. The call
stays the host's under its trace-time guard (a stand-in's function of
constants is a static test), is never traced while the guard holds, fails
by name if it ever does, and the note names it. Refusing the whole step
for a check that never runs was the alternative.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cne4hrGgYqhTMzVgzJtXYd
…and-in does

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cne4hrGgYqhTMzVgzJtXYd
The branch pruning took any expression statement for a log line the
anchor had left as a string and dropped the branch; a bare call -- a
procedure the port left on the host -- went with it, silently. Only
pass and a bare constant are inert.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cne4hrGgYqhTMzVgzJtXYd
…guard too

advance_clubb_core's parameterization check takes gr and err_info, so the
call reaches the flat-callee rewrite, which refused when the companion's
port had no kernel for it; the same host-call policy applies there.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cne4hrGgYqhTMzVgzJtXYd
The unpacks that follow a call the port left on the host read the
anchor's _out under the same guard; the elided-buffer entry of an earlier
call must not answer for them.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cne4hrGgYqhTMzVgzJtXYd

@chenyueqi chenyueqi left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review of #27 against two questions: generic or CLUBB-specific, and conflicts with existing rules on main (including #29/#30 merged today).

Genericity: every CLUBB mention in the engine (23 in transform, ~35 in fortran/oracle/verify) is a docstring or comment citing the motivating case; nothing branches on a CLUBB identifier. The rules (loop completion value, typed real/real fold, ordered SUM, max/min in bounds, optional presence forwarding, elementwise constructor arithmetic, allocation-by-any-name, generic-by-arity) are Fortran-semantics rules. Fine. (The KINDS_64 domain-name list this branch carries comes from #22; commented there.)

Existing rules: bitexact.py is main plus four additive hunks; #30's redraw loop, moved-extents rule and NaN rule are intact line for line. The typed fold lives in the tree evaluator, #30's packed constants and aliases in the token route; they never both decide a /. dump_replay keeps 9b2f515's int32-for-all-integer rule. No test deleted to hide a behavior change; the flipped rwset test (call refill(b%q, n) no longer reads q) is correct since q is a component, not a scope variable, and a real variable named q was never keyed on.

Three things to fix before merge, inline: the numba backend now emits an undefined _f_vsum; the stubbed-module call went from a recorded refusal to a runtime-only raise absent from deferred; left_to_module turns a refusal into an assumption that the recorded reference run can violate. Two smaller ones: record.py silently narrowed which dummies are recorded as outputs, and _allocation_bounds keys on component name alone across the whole file.

One note for the ledger rather than this PR: the for...else completion-value rule is line-order based (a redefinition in one if-arm hides a read in the sibling arm; a read before the loop inside an enclosing loop's next iteration; an index that is a dummy read by the caller) and falls back to Python's last-value semantics in those cases without refusing. That is main's behavior too, and it is documented nowhere (no hit for "completion value" / "one past" in docs). Worth a line in disclosure-ledger.md.

Also: test_loud_refusals.py swaps max(n, 2) for mod(n, 3) + 2 in the fixture because max in a bound is now accepted; the refusal test still exercises an unresolvable bound, but no test now asserts that a max bound is accepted.

"product": "np.prod",
"size": "np.size",
"sum": "np.sum",
"sum": "_f_vsum",

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The numba emitter reuses numpy.expressions.Expressions (numba/emitter.py:47, NumbaExpressions(Expressions)), so it picks up this mapping too, but numba/runtime.py defines and exports _f_vdot only, no _f_vsum. Any SUM in a numba candidate is now a NameError where main emitted np.sum (njit-supported). No numba test covers SUM. Either add _f_vsum to the numba runtime or let NumbaExpressions keep np.sum.

# ``if ( method == lapack )`` with it, and the candidate
# raised on the path the run *does* take.
reason = f"{name}: procedure of a stubbed module, not ported"
return [f"{pad}raise NotImplementedError({reason!r})"]

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Keeping the surrounding branch is the right call, but this line is returned as a rendered statement, so the block's status is mechanical and it never reaches candidate.deferred (translate.py collects only agent_queue); bitexact then judges the subprogram as if fully ported. On main the block was NoRule -> deferred -> counted in the evidence. git grep 'not ported' finds only this raise. Suggest recording it, e.g. a notes entry like the JAX side's host_calls in tree.py, so a reader of the verdict sees that a procedure was left unported rather than discovering it as a runtime failure on another input.

# No ``use`` reaches it, so no adapter sets it: both sides
# run with the module's own default, and the plan says so.
seen.add(name)
plan.left_to_module.append(f"{module}%{name}")

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment says "both sides run with the module's own default", but the recorded reference run may have set this variable through a public setter before the step: CLUBB's set_clubb_debug_level sets exactly error_code%clubb_debug_level. The plan records it under left_to_module, so it is not silent, but this turns what was a refusal into an assumption about the reference run. At minimum the flat plan could check the module for a public procedure that writes the variable and refuse when one exists.

# The plain OUT and INOUT dummies are outputs too (CLUBB's advance_*
# return wp2, wp3, ... beside what they write into the objects).
for a in originals:
if DERIVED.match(str(a["dtype"])) or a["intent"] not in ("OUT", "INOUT"):

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Before this change every non-derived array dummy with intent other than IN was recorded as an output, including undeclared intent (which extract spells as UNKNOWN). Now only OUT/INOUT are, with scalars added. An undeclared-intent array the routine writes is no longer captured, so the gate has no reference for it. The direction is fail-closed, but the narrowing is not in the description and no test covers the UNKNOWN case.

body = text[start.end() : at]
for item in COMPONENT_ALLOCATION.finditer(body.replace("&", " ")):
axes = [b.strip() for b in item.group(3).split(",")]
out.setdefault(item.group(2).lower(), axes)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_allocation_bounds now scans every allocate( in the file and keys by component name alone, so two types in one file with a same-named component get the first one's bounds. The previous this%-only scan bounded that collision. Keying on the object's dummy name (which the setup routine gives) as well would keep it bounded.

chenyueqi and others added 2 commits September 4, 2026 21:58
… ones included

The anchor's return tuple has one per dummy (an absent optional's is _)
and the unpacks index it by position; CLUBB's stats = _out[3] after
calc_brunt_vaisala_freq_sqd names an optional INOUT object the list had
left out.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cne4hrGgYqhTMzVgzJtXYd
…mmy is absent

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cne4hrGgYqhTMzVgzJtXYd
@chenyueqi

Copy link
Copy Markdown
Member Author

The places where this branch turns a refusal on main into a silent fallback are recorded apart from the review, in #32 (rows 7-10), so each is ruled on rather than merged by omission. A row there closes when the rule refuses again, records what it assumed in the evidence, or takes the assumption from the extension's configuration.

@chenyueqi

Copy link
Copy Markdown
Member Author

Second sweep, this pattern only: rows 15-21 of #32 are this branch's (the early-return flag in a loop body, the constructor token pass without integer division, the character component and the attribute-declared public variable both left to the module, the static-test and host-call assumptions in the JAX port, the stub-module names recorded as procedures).

chenyueqi and others added 16 commits September 4, 2026 22:49
A check the port left on the host, or a kernel call whose result nothing
binds, under a guard on the data: the branch binds nothing and nothing in
it could run under a tracer, so the lowering carries nothing rather than
refusing the kernel (advance_clubb_core's parameterization check under
any(err_code == fatal)).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cne4hrGgYqhTMzVgzJtXYd
After a statement that may have returned, _guard_after_returns wrapped
each following statement in its own `if not _ret`. A call whose result
lands in the _out buffer (never a carry) was then separated from its
unpacks by a lax.cond with no carried effects, and the kernel fell back
to the host. Wrap the rest of the block once instead, recursively, so
the buffer and its unpacks share a branch. Loop-return test gains an
elemental two-output call after the returning loop.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cne4hrGgYqhTMzVgzJtXYd
A kernel takes a dummy object apart into components, but its body may
still hand the object whole to something on the host: a framework
stand-in's query on it, a host-kept check under its guard. The name was
never bound in the kernel (UnboundLocalError on the first such use, in
the whole-step port). Rebuild it once at entry from the components, the
way the NumPy flat wrapper does before its call; absent optionals stay
None so present() stays false.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cne4hrGgYqhTMzVgzJtXYd
- a loop whose body lowered to nothing (statistics sampling dropped by
  its stand-in) is dropped, not refused for carrying nothing
- a logical scalar dummy is static under jit: a configuration switch,
  whose dead branch is never traced
- a named or computed stride (a grid's direction, +-1 at run time):
  trips counted by the runtime's _f_trips, the index remapped
- an output the anchor discards (_ = _out[k]) is dropped from the
  elided buffer instead of refused for having no actual
- an optional object the caller leaves out is absent in the callee:
  None in its component slots, its outputs dropped; a call that leaves
  out every (optional) object still reaches the flat function
- a callee's extent argument (an allocatable component sized by an
  allocating routine's dummy) is the caller's own extent argument for
  that axis, else that axis of the caller's component

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cne4hrGgYqhTMzVgzJtXYd
- a break inside a lowered loop is a flag the loop carries: set where
  the break was, it guards the rest of that trip and every later one;
  the DO variable's value at the exit is kept (int32) and rebound after
  the loop; a for-else (the DO ran to completion) runs under the flag
  inverted, its completion value cast to the index's dtype
- static loops scan over int32 indices (Fortran's default integer)
  instead of fori_loop's default-int scan, so a store of the index into
  an integer local or dummy keeps its dtype across a lax.cond
- a subprogram of the module the port could not emit, called from one
  it can, stays on the host (_host.<name>) under the anchor's guard and
  is named in the notes' host_calls, instead of delegating its caller
  and the callers above it; build_module returns those calls
- _lo_n/_st_n hoisted stride bounds are never carries

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cne4hrGgYqhTMzVgzJtXYd
A slice whose bounds trace (the loop index, through locals assigned
k -+ dir * n in scope) but whose length does not becomes the index
lo + arange(trips) * step -- a gather of static length at a traced
offset, a scatter when stored -- with the trip count the runtime's
_f_trips over static atoms, a Python int at trace time. The mask rules
keep the slices whose length depends on a traced bound.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cne4hrGgYqhTMzVgzJtXYd
Sequence association: the anchor spells a 2-d right-hand side handed to
a solver declared over three axes as np.reshape(rhs, (n, m, 1),
order='F'). The kernel's output for that dummy is reshaped to the
array's own shape, the same order, and rebinds it -- a reshape is no
store target, and the anchor's copy-out spelling does not change that.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cne4hrGgYqhTMzVgzJtXYd
A debug print under a level check, statistics under their switch: the
arms lower to nothing and the static Python if was emitted with no
body, a SyntaxError that took the whole emitted module down. Nothing in
both arms is no branch; nothing in one arm is pass.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cne4hrGgYqhTMzVgzJtXYd
iipdf_type == _mod.IIPDF_ADG1: pdf_closure picks its PDF by a constant
of model_flags spelled through the module alias. The static-test
predicate (and the tree's static-expression one) took only the bare
upper-case spelling; the dispatch was a lax.cond chain, every arm
traced, and the arm the run never takes carried a host driver's record
object into the cond. Static now, the dead arm is never traced.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cne4hrGgYqhTMzVgzJtXYd
…s fallback

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cne4hrGgYqhTMzVgzJtXYd
The expression mapping spells .not./.and./.or. as jnp.logical_*; under
jit those stage a constant into a tracer that no Python if can
convert (interpolation's .not. l_quintic_poly_interp took the whole
step down). The Python form of a static branch is now spelled with
Python's not/and/or, _f_concrete decides on the leaves (the
comparisons, the names), and the static predicate sees through the
mapped calls so a negated static comparison is static too.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cne4hrGgYqhTMzVgzJtXYd
… are its lowered stores

A DO WHILE under an IF inside a DO (interpolation's grid search): the
exit flag is a carry of the enclosing cond and needs a value before the
branch, at the top of the function like the goto-region flags, with the
one beside the loop resetting it on entry. And the loop's carries are
what its lowered body stores: a subscript store is a store to its base
only once lowered, and an inner loop's break flag and kept index are
the body's own.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cne4hrGgYqhTMzVgzJtXYd
…ried by enclosing loops

advance_helper's window search: a DO with an EXIT under an IF inside
the column loop. The flag and kept index initialized beside their loop
were carries of the enclosing cond with no value before the branch;
they now start at the top of the function (False, index 0) and every
loop enclosing a store of them carries them.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cne4hrGgYqhTMzVgzJtXYd
A DO variable's completion value, max(0, (ub - lb) // step), traces
when the bounds do (an inner loop over the outer index): Python's max
on a tracer is a boolean conversion. _f_pymax/_f_pymin keep Python
scalars Python (a static bound stays static) and fold tracers with
jnp.maximum/minimum.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cne4hrGgYqhTMzVgzJtXYd
jax.numpy has no erf; the math-to-jnp mapping spelled jnp.erf for the
anchor's scalar math.erf (CLUBB's cloud fraction). Both spellings now
reach _f_verf/_f_verfc, which import jax.scipy.special.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cne4hrGgYqhTMzVgzJtXYd
advance_xm_wpxp counts its right-hand sides under switches and a local
derived from them, then sizes its solver arrays by the count. A local
whose stores are all trace-time expressions -- at the top level or
under branches decided at trace time, logical forms included -- is
concrete: the tree tells the backend on a marker it strips, its
branches are Python ifs, its stores are never strengthened to jnp
scalars and are spelled with Python logic, and zeros((n, nrhs)) sees a
Python int. Logical scalar dummies are static in the tree as in the
backend.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cne4hrGgYqhTMzVgzJtXYd
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:emitter the NumPy translation (statements, expressions, parameters) area:flatten flat adapters for derived-type interfaces (flatten, recorder, plans) area:jax the JAX backend and tree port area:verifier the gates: static.rwset, differential.bitexact/tolerance, oracles enhancement New feature or request found-by:clubb surfaced by the recast-clubb extension on CLUBB_core

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant